Image-based indexing in a network-based marketplace

ABSTRACT

A method and system to index and search item listings in a network-based publication system, such as a network-based marketplace. For example, a listing query may be received from a user, the listing query including a query image. The query image may then be processed to generate a plurality of index sets. Each index set of the plurality of index sets may correspond to a respective portion of the query image and may comprise a set of index values. Each index value of the set of index values may be indicative of an image attribute, such as light intensity, for a respective sub-portion of the corresponding portion of the query image. At least one result image from a database of listing images may be identified by comparing the plurality of index sets for the query image to pre-compiled index sets in an index database. Each pre-compiled index set may be associated with at least one listing image which, in turn, may be associated with an item listing on a network-based marketplace.

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.12/406,016, filed on Mar. 17, 2009, the benefit of priority of which isclaimed hereby, and which is incorporated herein by reference in itsentirety.

BACKGROUND

As the use of network-based publication systems and marketplaces, suchas on-line commerce services or auction services expands, and the volumeof item listings in such applications increases, the speed, ease andconvenience with which information can be retrieved from suchmarketplaces increases in importance to customers.

Item listings in such network-based marketplaces typically includedetails of a particular item which is e.g. up for sale or auction andthese details are typically stored in text format and include adescription of the item together with other information, such as theprice, useful to a potential buyer in assessing the item listing, inaddition, item listings often include visual material related to theitem, typically in the form of a photograph, drawings or video clips.

The use of images in this context has customarily been limited to theprovision of information about item listings to customers, but it wouldbe useful to employ images associated with item listings for additionalpurposes, such as for image-based searching or for the automaticidentification of images for fraud protection purposes.

BRIEF DESCRIPTION OF DRAWINGS

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings in which:

FIG. 1 is a block diagram illustrating a publication system in theexample form of network-based marketplace system according to an exampleembodiment.

FIG. 2 is a diagrammatic representation of marketplace and paymentapplications which may form part of the example embodiment of FIG. 1.

FIG. 3 is a diagrammatic representation of functional models anddatabases which may form part of the example embodiment of FIG. 1.

FIG. 4 is a flow chart illustrating an example method to index a listingimage according to an example embodiment.

FIG. 5 is a flow chart illustrating an example method to perform animage-based search in the example embodiment of FIG. 1.

FIG. 6 is a schematic view of operations which may be performed on animage for indexing thereof according to an example embodiment.

FIG. 7 is a schematic view of operations which may be performed on animage for indexing thereof according to another example embodiment.

FIG. 8 is a schematic of operations which may be performed on an imagefor indexing thereof according to yet another example embodiment.

FIG. 9 is a schematic of operations which may be performed on an imagefor indexing thereof according to yet a further example embodiment.

FIG. 10 is a block diagram of a machine in the example form of acomputer system within which a set of instructions, for causing themachine to perform any one or more of the methodologies discussedherein, may be executed.

FIG. 11 is a diagrammatic view of a data structure according to anexample embodiment of a network-based marketplace.

DETAILED DESCRIPTION

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of some example embodiments. It will be evident, however,to one skilled in the art that the present invention may be practicedwithout these specific details.

In an example embodiment, there is provided a system and a method toindex images associated with item listings in a network-basedmarketplace, so that subsequent search or comparison operations areperformed on index data instead of on base images stored in an imagedatabase. Images in the image database are thus indexed by parsing orprocessing the images for feature extraction. The feature extraction mayinclude generating multiple sets of index values associated with eachimage. Each set of index values is also referred to as a feature vectoror an image fingerprint.

The index sets are associated with their respective images in aso-called reverse indexing arrangement, in which the index databasecontains a single entry for each unique index set or feature vector,with all the images that contain that particular index set or featurevector being listed against the entry.

When, for instance, a query image is subsequently presented in order tosearch for identical or similar images in the image database, the queryimage is itself parsed or processed to generate multiple feature vectorsor sets of index values for the query image. The index sets for thequery image are thus generated in a similar manner to the generation ofindex sets for the index database.

To identify images similar to the query image, all of the query image'sindex sets are compared to index sets in the index database. Whencommonality is identified between a query image index set and a databaseindex vector, a hit count is incremented for each of the imagesassociated with that index set in the index database. The databaseimages are ranked in order of hit count, so that the result of thesearch are those item listings whose images have index sets showing thegreatest commonality with the index sets of the query image.

In an example embodiment, the index sets are generated by firstperforming an edge detection algorithm on the image and then normalizingthe image. Thereafter, the image is divided into a raster of cells orsub-portions at a resolution which is usually significantly lower thanits native pixel resolution, so that each cell or sub-portion comprisesmany image pixels. The raster may typically be a 10 by 10 grid of cells.Next, an index value is assigned to each cell based on an imageattribute, typically a light intensity value, of the underlying imagepixels. The result is therefore a grid of index values. Thereafter, setsof index values (i.e. feature vectors) are created from respectiveoverlapping portions of the grid of index values. This process can beexplained as a sliding window that is moved over the grid to isolate twodimensional selections or portions of index values at a time, theportions overlapping and covering the entire grid. The index values ineach selection or portion together form one feature vector or set ofindex values. In an example embodiment, the moving window is three bythree cells in size, so that each feature vector is constituted by nineindex values.

The system and method thus provides for image comparison in apublication system, such as a network-based marketplace, which requiressignificantly less processing power, and is therefore faster than imagecomparison based on native image data, while returning results withrelatively high accuracy. The method is furthermore relatively robustand insensitive to scaling and resolution loss.

Architecture

One example embodiment of a distributed network implementing image-basedindexing for item listings in a network-based marketplace is illustratedin the network diagram of FIG. 1, which depicts a system 10 using aclient-server type architecture. A commerce platform, in the exampleform of a network-based marketplace platform 12, provides server-sidefunctionality, via a network 14 (e.g., the Internet) to one or moreclients. As illustrated, the platform 12 interacts with a web client 16executing on a client machine 20 and a programmatic client 18 executingon a client machine 22. In one embodiment, web client 16 is a webbrowser, but it may employ other types of web services.

Turning specifically to the network-based marketplace platform 12, anApplication Program Interface (API) server 24 and a web server 26 arecoupled to, and provide programmatic and web interfaces respectively to,one or more application servers 28. The application servers 28 host oneor more marketplace applications 30 and payment applications 32. Theapplication servers 28 are, in turn, shown to be coupled to one or moredatabases servers 34 that facilitate access to a number of databases, inparticular an item listing database 35, an image database 36, and anindex database 37. The item listing database 35 stores data indicativeof item listings for items which are offered for sale or auction on theplatform 12. Each item listing includes, inter alia, a text descriptionof the relevant item and metadata categorizing the item. The imagedatabase 36 includes images associated with respective item listings inthe item listing database 35. The images in the image database 36 may bestandard format image files such as JPEG files. The index database 37contains index data relating to images in the image database to permitimage-based searching of the image database 36. The format of index datain the index database is described in more detail below.

The marketplace applications 30 provide a number of marketplacefunctions and services to users that access the marketplace platform 12.The payment applications 32 likewise provide a number of paymentservices and functions to users. The payment applications 32 may allowusers to quantify for, and accumulate, value (e.g., in a commercialcurrency, such as the U.S. dollar, or a proprietary currency, such as“points”) in accounts, and then later to redeem the accumulated valuefor products (e.g., goods or services) that are made available via themarketplace applications 30. While the marketplace and paymentapplications 30 and 32 are shown in FIG. 1 to both form part of thenetwork-based marketplace platform 12, it will be appreciated that, inalternative embodiments, the payment applications 32 may form part of apayment service that is separate and distinct from the marketplaceplatform 12.

Further, while the system 10 shown in FIG. 1 employs a client-serverarchitecture, the present invention is of course not limited to such anarchitecture, and could equally well find application in a distributed,or peer-to-pee, architecture system. The various marketplace and paymentapplications 30 and 32 could also be implemented as standalone softwareprograms, which do not necessarily have networking capabilities.

The web client 16, it will be appreciated, accesses the variousmarketplace and payment applications 30 and 32 via the web interfacesupported by the web server 26. Similarly, the programmatic client 18accesses the various services and functions provided by the marketplaceand payment applications 30 and 32 via the programmatic interfaceprovided by the API server 24. The programmatic client 18 may, forexample, be a seller application (e.g., the TurboLister applicationdeveloped by eBay Inc., of San Jose, Calif.) to enable sellers to authorand manage listings on the marketplace platform 12 in an off-linemanner, and to perform batch-mode communications between theprogrammatic client 18 and the network-based marketplace platform 12.

FIG. 1 also illustrates a third party application 38, executing on athird party server machine 40, as having programmatic access to thenetwork-based marketplace via the programmatic interface provided by theAPI server 24. For example, the third party application 38 rimy,utilizing information retrieved from the network-based marketplaceplatform 12, support one or more features or functions on a websitehosted by the third party. The third party website may, for example,provide one or more promotional, marketplace or payment functions thatare supported by the relevant applications of the network-basedmarketplace platform 12.

FIG. 2 is a block diagram illustrating multiple marketplace and paymentapplications 30 and 32 that, in one example embodiment, are provided aspart of the network-based marketplace platform 12. The marketplaceplatform 12 may provide a number of listing and price-setting mechanismswhereby a seller may list goods or services for sale, a buyer canexpress interest in or indicate a desire to purchase such goods orservices, and a price can be set for a transaction pertaining to thegoods or services. To this end, the marketplace applications 30 areshown to include at least one publication application 41 and one or moreauction applications 44 which support auction-format listing and pricesetting mechanisms (e.g., English, Dutch, Vickrey, Chinese, Double,Reverse auctions etc.). The various auction applications 44 may alsoprovide a number of features in support of such auction-format listings,such as a reserve price feature whereby a seller may specify a reserveprice in connection with a listing and a proxy bidding feature whereby abidder may invoke automated proxy bidding.

A number of fixed-price applications 46 support fixed-price listingformats (e.g., the traditional classified advertisement-type listing ora catalogue listing) and buyout-type listings. Specifically, buyout-typelistings (e.g., including the Buy-It-Now (BIN) technology developed byeBay Inc., of San Jose, Calif.) may be offered in conjunction with anauction-format listing, and allow a buyer to purchase goods or services,which are also being offered for sale via an auction, for a fixed-pricethat is typically higher than the starting price of the auction.

Store applications 48 allow sellers to group their listings within a“virtual” store, which may be branded and otherwise personalized by andfor the sellers. Such a virtual store may also offer promotions,incentives and features that are specific and personalized to a relevantseller.

Reputation applications 50 allow parties that transact utilizing thenetwork-based marketplace platform 12 to establish, build and maintainreputations, which may be made available and published to potentialtrading partners. Consider that where, for example, the network-basedmarketplace platform 12 supports person-to-person trading, users mayhave no history or other reference information whereby thetrustworthiness and credibility of potential trading partners may beassessed. The reputation applications 50 allows a user, for examplethrough feedback provided by other transaction partners, to establish areputation within the network-based marketplace platform 12 over time.Other potential trading partners may then reference such a reputationfor the purposes of assessing credibility and trustworthiness.

Personalization applications 52 allow users of the marketplace platform12 to personalize various aspects of their interactions with themarketplace platform 12. For example a user may, utilizing anappropriate personalization application 52, create a personalizedreference page at which information regarding transactions to which theuser is (or has been) a party may be viewed. Further, a personalizationapplication 52 may enable a user to personalize listings and otheraspects of their interactions with the marketplace and other parties.

In one embodiment, the network-based marketplace platform 12 may supporta number of marketplaces that are customized, for example, for specificgeographic regions. A version of the marketplace may be customized forthe United Kingdom, whereas another version of the marketplace may becustomized for the United States. Each of these versions may operate asan independent marketplace, or may be customized (or internationalized)presentations of a common underlying marketplace.

Navigation of the network based-marketplace may be facilitated by one ormore navigation applications 56. For example, a keyword searchapplication 57 enables keyword searches of listings published via themarketplace platform 12. Similarly, an image search application 59enables an image-based search of item listings published via themarketplace platform 12. To perform an image-based search, a user willtypically submit a query image, whereupon the image search application59 may compare the query image to images in the image database toproduce a result list of item listings based on a similarity rankingbetween the query image and the images associated with the respectiveitem listings. The comparison ranking is established by parsing orprocessing the query image to provide index data, and thereaftercomparing the query image's index data to pre-compiled index data forthe listing images, as described in more detail below. A browseapplication allows users to browse various category, catalogue, orinventory data structures according to which listings may be classifiedwithin the marketplace platform 12. Various other navigationapplications may be provided to supplement the search and browsingapplications.

In order to make listings, available via the network-based marketplace,as visually informative and attractive as possible, as well as to enableimage-based searching, the marketplace applications 30 may include oneor more imaging applications 58, which users may use to upload imagesfor inclusion within listings. Images thus uploaded are stored in theimage database 36, each image being associatively linked to at least oneitem listing in the item listing database 35. One of the imagingapplications 58 also operates to incorporate images within viewedlistings. The imaging applications 58 may also support one or morepromotional features, such as image galleries that are presented topotential buyers. For example, sellers may pay an additional fee to havean image included within a gallery of images for promoted items.

The marketplace platform 12 may also include an image indexingapplication 61 to parse or process images uploaded via the imageapplication 58, as well as to parse or process query images submittedvia the image search application 59. The result of processing images bythe image indexing application 61 is index data which is stored in theindex database 37. Particular processes for indexing images, as well asthe format of index data, are discussed in/no/e detail below.

Listing creation applications 60 allow sellers conveniently to authorlistings pertaining to goods or services that they wish to transact viathe marketplace platform 12, and listing management applications 62allow sellers to manage such listings. Specifically, where a particularseller has authored and/or published a large number of listings, themanagement of such listings may present a challenge. The listingmanagement applications 62 provide a number of features (e.g.,auto-relisting, inventory level monitors, etc) to assist the seller inmanaging such listings. One or more post-listing management applications64 also assists sellers with a number of activities that typically occurpost-listing. For example, upon completion of an auction facilitated byone or more auction applications 44, a seller may wish to leave feedbackregarding a particular buyer. To this end, a post-listing managementapplication 64 may provide an interface to one or more reputationapplications 50, so as to allow the seller conveniently to providefeedback regarding multiple buyers to the reputation applications 50.

Dispute resolution applications 66 provide mechanisms whereby disputesarising between transacting parties may be resolved. For example, thedispute resolution applications 66 may provide guided procedures wherebythe parties are guided through a number of steps in an attempt to settlea dispute. In the event that the dispute cannot be settled via theguided procedures, the dispute may be escalated to a third partymediator or arbitrator.

A number of fraud prevention applications 68 implement various frauddetection and prevention mechanisms to reduce the occurrence of fraudwithin the marketplace. One of the fraud prevention applications 68 mayinclude automatic image comparison, by use of index data produced by theimage indexing application 61 and stored in the index database 37. Suchimage comparison may be used by the fraud prevention application 68automatically to detect listing images similar to the query image, andto alert a fraud assessor to such image listings, so that the humanassessor can examine the identified item listing for assessing whetheror not the identified item listing is a fraudulent listing.

Messaging applications 70 are responsible for the generation anddelivery of messages to users of the network-based marketplace platform12, such messages for example advising users regarding the status oflistings at the marketplace (e.g., providing “outbid” notices to biddersduring an auction process or providing promotional and merchandisinginformation to users).

Merchandising applications 72 support various merchandising functionsthat are made available to sellers to enable sellers to increase salesvia the marketplace platform 12. The merchandising applications 72 alsooperate the various merchandising features that may be invoked bysellers, and may monitor and track the success of merchandisingstrategies employed by sellers.

The network-based marketplace platform 12 itself, or one or more partiesthat transact via the marketplace, may operate loyalty programs that aresupported by one or more loyalty/promotions applications 74. Forexample, a buyer may earn loyalty or promotions points for eachtransaction established and/or concluded with a particular seller, andbe offered a reward for which accumulated loyalty points can beredeemed.

FIG. 3 is a high-level entity-relationship diagram, illustrating therelationship between the databases 35 to 37 and several functionalmodules forming part of the applications 30 and 32. The system includesa receiving module 80, which may form part of the image searchapplication 59 (FIG. 2), for receiving a listing query which includes aquery image. Query images which may be submitted to the receiving module80 are typically electronic image files in a standard format, such asJPEG, GIF, TIFF, etc.

The receiving module 80 is operatively connected to a processing module86, which may form part of the image indexing application 61, forprocessing images to generate index data for the images. An examplemethod of processing images by the processing module 86 to generate theindex data is described in more detail below. An image listing module82, which may form part of the listing creation application 60 of FIG.2, is operatively connected to the processing module 86 to communicateto the processing module 86 images forming part of newly created itemlistings. The system further includes a database crawler 84 which servesto interrogate the listing image database 36 in order to identify imagesin the listing image database 36 which have not yet been processed bythe image indexing application 61, and for which there is thus noassociated index data in the index database 37. It will be appreciatedthat indexing of images in accordance with an example embodiment may incertain instances be implemented in a network-based marketplace havinglarge numbers of existing listing images for which there are, of course,no index data. To permit image-based searching of the listing imagedatabase 36 through the use of index data, the database crawler maycontinually locate and submit un-indexed legacy images in the listingimage database 36 and pass them to the processing module for imaging. Itshould be noted accordingly that although image database 36 isillustrated in the figures by a single element, the image database 36may be provided by any number of databases distributed through anetwork.

The processing module 86 is configured to parse or process imagessubmitted to it to generate index data in the form of a number of afeature vector or index sets 88 for each image. In an exampleembodiment, each index set 88 comprises a sequence of nine index values,as shown in FIG. 3. These index sets 88 are also referred to as featurevectors or image fingerprints, and the terms “index set”, “set of indexvalues”, and “feature vector” are used interchangeably in this documentto refer to the results of indexing of images by the processing module86.

The processing module 86 is in communication with the index database 37,to communicate index data generated by the processing module 86 to theindex database 37 for storage. In an example embodiment, index data isrelated in the index database 37 to images stored in the listing imagedatabase 36 in a reverse-indexing format. The data format in the indexdatabase 37 may be in a spatial data structure, such as atwo-dimensional k-d tree, to facilitate searching of the database 37. Asillustrated in FIG. 3, the index database has stored therein a singleentry for each unique feature vector 88, with each entry listing all ofthe database images which share that feature vector 88. For example withreference to FIG. 3, it will be understood that one of the featurevectors 88 produced by processing of an image with the filename IMG#23,stored in the image database 36, was [0, 0, 0, 2, 6, 12, 15, 13, 15],and that this feature vector was also one of the feature vectors 88produced by processing of IMG#3012. It should be appreciated thatmultiple feature vectors 88 are generated by the processing of eachimage, and that a particular image will thus be listed in the indexdatabase against each of the multiple feature vectors 88 thus produced.For instance, the example given in FIG. 3 shows that the results ofindexing of IMG#3012 included feature vector [0, 0, 0, 2, 6, 12, 15, 13,15] and feature vector [4, 10, 17, 13, 9, 0, 0, 13, 15].

Each entry in the index database 37 is linked to at least one image inthe listing image database 36. Further, the images stored in the imagedatabase 36 are linked to associated item listings in the item listingdatabase 35. As discussed above, each item listing may compriseinformation about a particular item which is offered for sale or auctionon the marketplace platform 12, such as a description of the item, atleast one category of the item, a brand name of the item, etc. In theexample embodiment, entries in the respective databases 35 to 37 arelinked by use of image filenames as linking data. A person skilled inthe art will appreciate that, in other embodiments, any appropriate datastructure, e.g. relational databases or tables, may be used to linkimages in the image database 36 to, on the one hand, respective itemlistings and to, on the other hand, index data in the index database 37.

A comparison module 90 is in communication with both the processingmodule 86 and the index database 37, to compare index data for a queryimage with the index data in the index database 37 for finding databaseimages similar to the query image. The comparison module, in use,produces a ranking of database images according to similarity to thequery image. This ranking is achieved by comparing the feature vectors88 of the query image, which is generated by the processing module 86,with feature vector entries in the index database 3 accordance with acomparison method which is described in more detail below.

The system 10 also includes a result module 92 for delivering to a userthe results of a search query which includes a query image. The resultmodule 92 is configured to return query search results as item listingsranked in order of the similarity of their respective images to thequery image, as determined by the comparison module 90. The searchresults may be delivered by the user in a format of the user's choosing,such as, for instance, via e-mail or in http format on a web browserapplication.

Flowcharts

FIG. 4 is a flowchart showing a method 100 for processing an image in anetwork-based marketplace in accordance with an example embodiment. Themethod 100 starts with the creation of an item listing, at block 102, bya user via the listing creation application 60 (FIG. 2) in customaryfashion. Creation of the item listing may include submission of an imagerelated to the image, typically being an image of the item which isoffered for sale or auction on the marketplace. The submitted image isreceived by the receiving module 80, at block 104.

The image may be an electronic image in standard file format, such asPEG, which comprises a raster of pixels. Each pixel may have hue,saturation, and intensity values, in conventional fashion. It will beappreciated that images which were submitted earlier and which arestored in the image database 36, but which have not been indexed, may beprovided to the processing module 86 by the database crawler 84 of FIG.3, so that operation 102 in FIG. 4 may instead comprise submitting anun-indexed image from the image database 36 for processing.

The submitted image is then processed, at block 122, to generate indexdata for enabling index searching of database images. Processing of theimage will be described at the hand of an example image 200 illustratedin FIG. 6. First, an edge detection and normalisation operation isperformed, at block 106, on the image 200, to produce a normalised edgeimage 202 (FIG. 6). Edge detection processing is well known in the artand any suitable edge detection algorithm may be employed in operation106. Normalisation of the image includes desaturation, so that thenormalised edge image 202 is a greyscale image. Normalization mayfurther include contrast stretching or dynamic range expansion, toachieve consistency in intensity ranges for images processed by theprocessing module 86. Furthermore, normalization of the image mayinclude re-alignment of the image 200 by automated correction of theimage's orientation.

The normalised edge image 202 is then partitioned or divided, at block108, into cells or sub-portions 207, to form a grid 204 (FIG. 6). In theexample embodiment, the resolution of the grid is ten by ten, so thatthe grid comprises ten rows of ten sub-portions or cells 207 each.Because the size of each block or cell 207 is considerably larger thanthat of the image's pixels, each cell 207 consists of a plurality ofpixels. It will be appreciated that the resolution of the grid maydiffer in other embodiments.

Thereafter, an index value 208 is assigned, at block 110, to each cell207 based on an image attribute of the underlying image pixels of thecell 207. In this example, the image attribute is intensity, typicallymeasured on a scale of 0 to 255, or alternatively 0 to 100, where apixel having a zero intensity is white and a pixel having intensityvalue of 255 (or 100, as the case may be) is black. The index value 208assigned to each cell 207 may thus be the average light intensity valueof the pixels constituting each cell 207. The output of operation 110 istherefore a ten-by-ten grid 206 of index values 208 based on theintensity values of the respective cells 207 (FIG. 6). The grid 206 canbe viewed as a two-dimensional histogram of the base image 200. It is tobe appreciated that the particular index values 208 and the particularindex sets 88 illustrated in FIGS. 3 and 6, and in other examples inthis document, are chosen arbitrarily for illustrative purposes and donot accurately reflect the underlying intensity values of theillustrated images.

In other example embodiments, other image attributes can be used as wellas or instead of the intensity value. For instance, colour values of thecells 207 may be calculated and indexed together with or instead of theintensity index values. An index value may for instance assigned to eachcell 207 based on the average hue of the cell's pixels. Instead,separate grids may be produced for red, green and blue colour spaces,and index values based on the intensity values of the respective coloursin the cells may be assigned to the cells.

At block 140, feature vectors or index sets 88 are compiled from thegrid 206. Compilation of the index sets 88 comprises iterativelyisolating portions 210 of the grid 206 and listing the index values 208in each portion 210 in sequence, to provide an index set 88. Compilationof the index sets 88 can thus be described as a sliding, overlappingmask or window 210 which is three-by-three index values in size, andwhich iteratively isolates all possible contiguous three-by-threeselections in the grid 206, to generate respective index sets 88. Eachindex set 88 thus comprises a sequence or vector of nine index values208. Although only two of these index sets 88 are shown in FIG. 6, itwill be appreciated that the results of index set compilation for aten-by-ten grid 206 will be 64 index sets.

The index sets 88 thus generated are incorporated in the index database37. As explained above with reference to FIG. 3, the index database 37comprises a single entry for each unique feature vector or index set 88,with all images which contain that index set 88 being listed in theentry. To this end, each index set 88 generated in operation 112 isprocessed by first establishing, at block 114, whether or not the indexdatabase 37 already includes an entry for that particular index set 88.If the determination at block 114 is in the affirmative, the image whichis the subject of current processing is linked, at block 118, to theexisting entry in the index database 37 by including the image filenamein the listing of images in the respective database entry, if, however,the determination is in the negative and there is not yet a databaseentry for the index set 88 under consideration, a new index set entry iscreated, at block 116, in the index database 37. It will be appreciatedthat such a new database entry will have only the current imageassociated with the particular index set 88. This database writingsequence is looped, at block 120, through all of the index sets 88generated at operation 112, therefore being performed 64 times in theexample embodiment having a ten-by-ten grid 206.

FIG. 5 is flowchart of a method 130 of image-based searching inaccordance with an example embodiment. The method 130 is initiated byuser submission of a query, at block 132, via the image searchapplication 59 of FIG. 2, and includes a query image on which the searchis to be based. The query image is again a digital image in a standardfile format. A user wishing, for instance, to search for item listingsin respect of a particular device may photograph the target device, forexample by use of a mobile phone with image capturing capabilities, andmay submit the image to the image search application instead of or inaddition entering text in the keyword search application 57.

The query image is processed, at block 122, by the processing module 86to generate index sets 88 for the query image in a manner identical toindexing of images in accordance with the method 100 of FIG. 5. Edgedetection and normalisation of the image is thus performed, at block106, whereafter the query image is partitioned in a grid of cells 207,at block 108. Then, index values 208 are assigned to each cell 207 basedon the intensity values of the underlying pixels of the cell 207, andindex sets 88 are compiled by use of the sliding, overlapping window 210method described above.

After generation of the query index sets 88 from the query image, thequery feature vectors or index sets 88 are compared to the index data inthe index database 37 to identify images similar or identical to thequery image. To this end, the comparison module 90 processes each of thequery index sets 88 in turn. The comparison module 90 steps or loops, atblock 146, through the index values 208 of a particular index set 88 tofind, at block 142, all index set entries in the index database 37 thatshare that index value 208. For each index entry identified as sharingthe particular index value 208 under consideration, a hit count isincremented for each of the images associated in the index database 37with the identified database entry. The database images are then rankedin descending order according to hit count.

In an example where the first query index set 88 is [24, 12, 13, . . . ,4], the first index value 208 is 24. If the index database 37 includesthe following entries:

1. [0, 0, 24, 16, 26, . . . ]=IMG#221, IMG#3224, IMG#6739

2. [36, 48, 18, . . . ]=IMG#644, IMG#2542

3. [24, 12, 0, 0, . . . ]=IMG#3224, IMG#2143,

the first iteration of operation 142, in respect of value 24, willresult in an increment in the hit count of the images in data entries 1and 3 above. The second iteration of operation 142, which will be inrespect of value 12, will result in the incrementing, at block 144, ofthe images in data entry 3 above. The results of looping through all theindex values 208 of query index set 88 shown above will be IMG#3224=3hits; IMG#2143=2 hits, with the remainder of the listed imagesregistering a single hit, apart from IMG#644 and IMG#2542 which willhave no hits registered against them.

After repeating operations 142 to 146 for all of the index values 208 ofone of the index sets 88, the process loops, at block 148, to the nextquery index set 88. Operations 142 to 148 are thus repeated until all ofthe index values of all of the query index sets 88 have been compared tothe index database entries, the hit counts being aggregated to provide aranking of images by hit count, at block 150.

In other embodiments, the comparison of index sets may include comparingall of the index values of query index sets with all of the values inthe respective pre-compiled index sets forming database entries in theindex database 37, to that a hit is registered only if there is completeoverlap between the index values of, on the one hand, the query indexset, and, on the other hand, the index values of the particular databaseentry. In yet further embodiments, the comparison of feature vectors orindex sets may include matching not only the values of the query indexsets to database entry index sets, but also matching the sequence ofindex values in the respective index sets. A hit will thus be registeredonly if the query index set matches a database entry's feature vector orindex set 88 exactly, in other words if both the values and the sequenceof the respective index sets are identical. To promote processing speedand efficiency when performing exact feature vector or index setmatching, the index value range may be reduced in scale, so that theindex values, for example, range in value from 0 to 10 instead of forexample, from 0 to 100.

Instead, or in addition, the comparison operation may include aweighting of the hit count based on the position of the respective indexsets in the image. In other words, hits may be assigned weights based onadjacency of the index sets in the image. Two matching index sets whichwere compiled from image portions or windows which are in adjacent oridentical locations in the grid may therefore result in a higherweighted hit, while a lower weighted hit may be registered if therespective image portions or windows are less adjacent.

As mentioned above, pre-complied feature vectors or index sets 88 may bestored in the index database in a data structure like a k-dimensionaltree, also known as a k-d tree. Comparison of a query index set orfeature vector may in such cases comprise performing a nearest neighboursearch in the k-dimensional tree.

The hit counts of the images are passed by the comparison module 90 tothe result module 92. The result module 92 then displays to the user, atblock 152, the results of the search. The search results are provided asa list of item listings extracted from the item listing database 35, thedisplayed item listing being the item listings linked to the top rankedimages, as identified by the comparison module 90.

The system 10 and methods 100, 130 described above provides foreffective image-based searching in the network-based marketplace.Indexing of the images in the image database 36 in accordance with thedescribed example embodiment permits similarity comparison of the queryimage with large numbers of database images without requiringprohibitive processing power or time.

In addition to use of the indexing method 100 for user-initiatedimage-based searching of the database 36, it may, in other embodiments,be used for fraud prevention applications in the network-basedmarketplace. In such embodiments, the fraud prevention application 68,shown in FIG. 2, may be provided with a query image representing anarticle which may be susceptible to fraud. Image comparison as describedabove with reference to FIG. 5 may then automatically be performed inresponse to the creation of new listings, so as to flag new itemlistings having images with a similarity rating or weighting, asdetermined by index data comparison, higher than a set threshold value.

In other embodiments, the processing operation 122 to generate indexsets 88 may differ in a number of aspects, some of which are describedwith reference to FIGS. 7-9. In one embodiment, illustrated in FIG. 7,the indexing method 100 includes producing a number of variations of asubject image 200, and processing each of the variations to produce aplurality of index sets 88 for each of the variations. In other words, asingle base image 200 is used to produce multiple image variations, andeach of the image variations is indexed and its index data is recordedin the index database as separate images, each of which is linked to acommon item listing. Identification of any one of these image variationsranking images by hit count, at block 150 in FIG. 5, will result in theassociated item listing being presented in results of an image-basedquery.

In another embodiment, such image variation may be performed uponsearching instead of, or in addition to, image variation duringindexing. In such case, a query image may thus be processed to producemultiple image variations, index sets 88 thereafter being generated foreach of the variations and being compared to the index database 37.

In the embodiment illustrated in FIG. 7, the subject image is cropped atthree different magnification levels to produce three edge imagevariations 220 to 224. Each of these edge image variations 220 to 224are then partitioned into sub-portions 207 to provide respective grids226 to 230. Although not illustrated explicitly in FIG. 7, the grids 226to 230 are then assigned index values 208, and index sets 88 arecompiled as described with reference to FIGS. 4 and 6, it will beappreciated that the number of magnification levels, and therefore thenumber of image variations 220 to 224 can be varied. In one embodiment,which is not illustrated, ten image variations based on varying croppingmagnifications may be produced.

In another embodiment, illustrated in FIG. 9, the subject image 200 isrotated or angularly displaced at three different angles to producethree edge image variations 250 to 254. These edge image variations 250to 254 are then partitioned into grids 260 to 264 for further processingto generate respective collections of index sets 88.

In yet another embodiment, illustrated in FIG. 8, the subject image isfirst subjected to edge detection and normalization, to produce anormalised edge image 202. Thereafter, the normalised edge image 202 ispartitioned at three different grid resolutions. In the exampleembodiment, the image 202 is partitioned at a 5×5 resolution to producea first grid 240; it is partitioned at a 10×10 resolution to produce asecond grid 242; and it is partitioned at a 20×20 resolution to producea third grid 244. Each of these grids is further processed to producerespective collections of index sets 88, and each of the variations240-244 may be recorded in the index database 37. It will be appreciatedthat each of the variations is linked to the common image listing, sothat identification of any of the variations 240 to 244 in animage-based search will result in return of an item listing associatedwith the subject image 200.

In another embodiment, the partitioning resolution for indexing may bedetermined by a category of the relevant item listing. For example, themethod may include categorising an item listing upon creation,determining the item listing's category before processing the image, atblock 108, and selecting the partitioning resolution based on the itemlisting category. For instance, apparel may be partitioned at a 10×10resolution, while electronic devices may be partitioned at a 15×15resolution.

It will further be appreciated that the system and methodology describedabove can be applied to video content as well as or instead of imagedata. The method may in such case include extracting images from videofiles, and processing the extracted images in accordance with theexample embodiments described herein. Image extraction from such videocontent may include automatically identifying scene changes in the videocontent, e.g. by comparison of successive frames or images in the videocontent, and extracting images or frames at the start of such scenechanges.

The example embodiments described herein address some of the technicalchallenges associated with effective processing of images linked to itemlistings. For instance, image databases of network-based marketplacesare often very large, comprising millions of images, so that the timeand/or processing power consumed by conducting a search or imagecomparison in the database is prohibitive. However, a comparison ofindex data generated for the query image with index data of databaseimages is considerably less resource intensive.

Modules, Components and Logic

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. A component is a tangible unitcapable of performing certain operations and may be configured orarranged in a certain manner. In example embodiments, one or morecomputer systems (e.g., a standalone, client or server computer system)or one or more components of a computer system e.g., a processor or agroup of processors) may be configured by software (e.g., an applicationor application portion) as a component that operates to perform certainoperations as described herein.

In various embodiments, a component may be implemented mechanically orelectronically. For example, a component may comprise dedicatedcircuitry or logic that is permanently configured (e.g., as aspecial-purpose processor) to perform certain operations. A componentmay also comprise programmable logic or circuitry (e.g., as encompassedwithin a general-purpose processor or other programmable processor) thatis temporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a componentmechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “component” should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired) or temporarily configured(e.g., programmed) to operate in a certain manner and/or to performcertain operations described herein. Considering embodiments in whichcomponents are temporarily configured (e.g., programmed), each of thecomponents need not be configured or instantiated at any one instance intime. For example, where the components comprise a general-purposeprocessor configured using software, the general-purpose processor maybe configured as respective different components at different times.Software may accordingly configure a processor, for example, toconstitute a particular component at one instance of time and toconstitute a different component at a different instance of time.

Components can provide information to, and receive information from,other components. Accordingly, the described components may be regardedas being communicatively coupled. Where multiple of such componentsexist contemporaneously, communications may be achieved through signaltransmission (e.g., over appropriate circuits and buses) that connectthe components. In embodiments in which multiple components areconfigured or instantiated at different times, communications betweensuch components may be achieved, for example, through the storage andretrieval of information in memory structures to which the multiplecomponents have access. For example, one component may perform anoperation, and store the output of that operation in a memory device towhich it is communicatively coupled. A further component may then, at alater time, access the memory device to retrieve and process the storedoutput. Components may also initiate communications with input or outputdevices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods described herein may be at least partiallyprocessor-implemented. For example, at least some of the operations of amethod may be performed by one or more processors orprocessor-implemented modules. The performance of some of the operationsmay be distributed among the one or more processors, not only residingwithin a single machine, but deployed across a number of machines. Insome example embodiments, the processor or processors may be located ina single location (e.g., within a home environment, an officeenvironment or as a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of theoperations may be performed by a group of computers (as examples ofmachines including processors), these operations being accessible via anetwork (e.g., the Internet) and via one or more appropriate interfaces(e.g., Application Program Interfaces (APIs).)

Electronic Apparatus and System

Example embodiments may be implemented in digital electronic circuitry,or in computer hardware, firmware, software, or in combinations thereof.Example embodiments may be implemented using a computer program product,e.g., a computer program tangibly embodied in an information carrier,e.g., in a machine-readable medium for execution by, or to control theoperation of, data processing apparatus, e.g., a programmable processor,a computer, or multiple computers.

A computer program can be written in any form of programming language,including compiled or interpreted languages, and it can be deployed inany form, including as a stand-alone program or as a module, subroutine,or other unit suitable for use in a computing environment. A computerprogram can be deployed to be executed on one computer or on multiplecomputers at one site or distributed across multiple sites andinterconnected by a communication network.

In example embodiments, operations may be performed by one or moreprogrammable processors executing a computer program to performfunctions by operating on input data and generating output. Methodoperations can also be performed by, and apparatus of exampleembodiments may be implemented as, special purpose logic circuitry,e.g., an field programmable gate array (FPGA) or an application-specificintegrated circuit (ASIC).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. Inembodiments deploying a programmable computing system, it will beappreciated that that both hardware and software architectures requireconsideration. Specifically, it will be appreciated that the choice ofwhether to implement certain functionality in permanently configuredhardware (e.g., an ASIC), in temporarily configured hardware (e.g., acombination of software and a programmable processor), or a combinationof permanently and temporarily configured hardware may be a designchoice. Below are set out hardware (e.g., machine) and softwarearchitectures that may be deployed, in various example embodiments.

Example Three-Tier Software Architecture

In some embodiments, the described methods may be implemented using oneof a distributed or non-distributed software application designed undera three-tier architecture paradigm. Under this paradigm, various partsof computer code (or software) that instantiate or configure componentsor modules may be categorized as belonging to one or more of these threetiers. Some embodiments may include a first tier as an interface (e.g.,an interface tier). Further, a second tier may be a logic (orapplication) tier that performs application processing of data inputtedthrough the interface level. The logic tier may communicate the resultsof such processing to the interface tier, and/or to a backend, orstorage tier. The processing performed by the logic tier may relate tocertain rules, or processes that govern the software as a whole. Athird, storage tier, may be a persistent storage medium, or anon-persistent storage medium. In some cases, one or more of these tiersmay be collapsed into another, resulting in a two-tier architecture, oreven anon-tier architecture. For example, the interface and logic tiersmay be consolidated, or the logic and storage tiers may be consolidated,as in the case of a software application with an embedded database. Thethree-tier architecture may be implemented using one technology, or, avariety of technologies. The example three-tier architecture, and thetechnologies through which it is implemented, may be realized on one ormore computer systems operating, for example, as a standalone system, ororganized in a server-client, peer-to-peer, distributed or some othersuitable configuration. Further, these three tiers may be distributedbetween more than one computer systems as various components.

Components

Example embodiments may include the above described tiers, and processesor operations about constituting these tiers may be implemented ascomponents. Common to many of these components is the ability togenerate, use, and manipulate data. The components, and thefunctionality associated with each, may form part of standalone, client,server, or peer computer systems. The various components may beimplemented by a computer system on an as-needed basis. These componentsmay include software written in an object-oriented computer languagesuch that a component oriented, or object-oriented programming techniquecan be implemented using a Visual Component Library (VCL), ComponentLibrary for Cross Platform (CLX), Java Beans (JB), Java Enterprise Beans(EJB), Component Object Model (COM), Distributed Component Object Model(DCOM), or other suitable technique.

Software for these components may further enable communicative couplingto other components (e.g., via various Application Programminginterfaces (APIs)), and may be compiled into one complete server,client, and/or peer software application. Further, these APIs may beable to communicate through various distributed programming protocols asdistributed computing components.

Distributed Computing Components and Protocols

Some example embodiments may include remote procedure calls being usedto implement one or more of the above described components across adistributed programming environment as distributed computing components.For example, an interface component (e.g., an interface tier) may formpart of a first computer system that is remotely located from a secondcomputer system containing a logic component (e.g., a logic tier). Thesefirst and second computer systems may be configured in a standalone,server-client, peer-to-peer, or some other suitable configuration.Software for the components may be written using the above describedobject-oriented programming techniques, and can be written in the sameprogramming language, or a different programming language. Variousprotocols may be implemented to enable these various components tocommunicate regardless of the programming language used to write thesecomponents. For example, a component written C++ may be able tocommunicate with another component written in the Java programminglanguage through utilizing a distributed computing protocol such as aCommon Object Request Broker Architecture (CORBA), a Simple ObjectAccess Protocol (SOAP), or some other suitable protocol. Someembodiments may include the use of one or more of these protocols withthe various protocols outlined in the Open Systems Interconnection (OSI)model, or Transmission Control Protocol/Internet Protocol (TCP/IP)protocol stack model for defining the protocols used by a network totransmit data.

A System of Transmission Between a Server and Client

Example embodiments may use the OSI model or TCP/IP protocol stack modelfor defining the protocols used by a network to transmit data. Inapplying these models, a system of data transmission between a serverand client, or between peer computer systems may for example includefive layers comprising: an application layer, a transport layer, anetwork layer, a data link layer, and a physical layer. In the case ofsoftware, for instantiating or configuring components, having a threetier architecture, the various tiers (e.g., the interface, logic, andstorage tiers) reside on the application layer of the TCP/IP protocolstack. In an example implementation using the TCP/IP protocol stackmodel, data from an application residing at the application layer isloaded into the data load field of a TCP segment residing at thetransport layer. This TCP segment also contains port information for arecipient software application residing remotely. This TCP segment isloaded into the data load field of an IP datagram residing at thenetwork layer. Next, this IP datagram is loaded into a frame residing atthe data link layer. This frame is then encoded at the physical layer,and the data transmitted over a network such as an internet, Local AreaNetwork (LAN), Wide Area Network (WAN), or some other suitable network.In some cases, internet refers to a network of networks. These networksmay use a variety of protocols for the exchange of data, including theaforementioned TCP/IP, and additionally ATM, SNA, SDI, or some othersuitable protocol. These networks may be organized within a variety oftopologies (e.g., a star topology), or structures.

Although an embodiment has been described with reference to specificexample embodiments, it will be evident that various modifications andchanges may be made to these embodiments without departing from thebroader spirit and scope of the embodiment. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense. The accompanying drawings that form a parthereof, show by way of illustration, and not of limitation, specificembodiments in which the subject matter may be practiced. Theembodiments illustrated are described in sufficient detail to enablethose skilled in the art to practice the teachings disclosed herein.Other embodiments may be utilized and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. This Detailed. Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

Data Structures

FIG. 11 is a high-level entity-relationship diagram of an exampleembodiment, illustrating various tables 700 that may be maintainedwithin the databases 35 to 37, and that are utilized by and support theapplications 30 and 32. A user table 702 contains a record for eachregistered user of the networked system 12, and may include identifier,address and financial instrument information pertaining to each suchregistered user. A user may operate as a seller, a buyer, or both,within the networked system 12. In one example embodiment, a buyer maybe a user that has accumulated value (e.g., commercial or proprietarycurrency), and is accordingly able to exchange the accumulated value foritems that are offered for sale by the networked system 12.

The tables 700 also include an items table 704 in which are maintaineditem records for goods and services that are available to be, or havebeen, transacted via the networked system 12. Each item record withinthe items table 704 may furthermore be linked to one or more userrecords within the user table 702, so as to associate a seller and oneor more actual or potential buyers with each item record.

The items table 704 may be connected to an image table which containsimages associated with the respective items or item listings in theitems table 704. The image table 720 is in turn connected to an indexdata table 730 which contains index data as described in detail above.

A transaction table 706 contains a record for each transaction (e.g., apurchase or sale transaction) pertaining to items for which recordsexist within the items table 704.

An order table 708 is populated with order records, each order recordbeing associated with an order. Each order, in turn, may be with respectto one or more transactions for which records exist within thetransaction table 706.

Bid records within a bids table 710 each relate to a bid received at thenetworked system 12 in connection with an auction-format listingsupported by an auction application 32. A feedback table 712 is utilizedby one or more reputation applications 50, in one example embodiment, toconstruct and maintain reputation information concerning users. Ahistory table 714 maintains a history of transactions to which a userhas been a party. One or more attributes tables 716 record attributeinformation pertaining to items for which records exist within the itemstable 704. Considering only a single example of such an attribute, theattributes tables 716 may indicate a currency attribute associated witha particular item, the currency attribute identifying the currency of aprice for the relevant item as specified in by a seller.

FIG. 10 shows a diagrammatic representation of a machine in the exampleform of a computer system 500 within which a set of instructions, forcausing the machine to perform any one or more of the methodologiesdiscussed herein, may be executed. In alternative embodiments, themachine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be aserver computer, a client computer, a personal computer (PC), a tabletPC, a set-top box (SIB), a Personal Digital Assistant (PDA), a cellulartelephone, a web appliance, a network router, switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

The example computer system 500 includes a processor 502 (e.g., acentral processing unit (CPU) a graphics processing unit (GPU) or both),a main memory 504 and a static memory 506, which communicate with eachother via a bus 508. The computer system 500 may further include a videodisplay unit 510 (e.g., a liquid crystal display (LCD) or a cathode raytube (CRT)). The computer system 500 also includes an alphanumeric inputdevice 512 (e.g., a keyboard), a cursor control device 514 (e.g., amouse), a disk drive unit 516, a signal generation device 518 (e.g., aspeaker) and a network interface device 520.

The disk drive unit 516 includes a machine-readable medium 522 on whichis stored one or more sets of instructions (e.g., software 524)embodying any one or more of the methodologies or functions describedherein. The software 524 may also reside, completely or at leastpartially, within the main memory 504 and/or within the processor 502during execution thereof by the computer system 500, the main memory 504and the processor 502 also constituting machine-readable media.

The software 524 may further be transmitted or received over a network526 via the network interface device 520.

While the machine-readable medium 522 is shown in an example embodimentto be a single medium, the term “machine-readable medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database, and/or associated caches and servers) thatstore the one or more sets of instructions. The term “machine-readablemedium” shall also be taken to include any medium that is capable ofstoring, encoding or carrying a set of instructions for execution by themachine and that cause the machine to perform any one or more of themethodologies described herein. The term “machine-readable medium” shaltaccordingly be taken to include, but not be limited to, solid-statememories, optical and magnetic media, and carrier wave signals.

Thus, a method and system to index images and to perform an image-basedsearch in a network-based marketplace have been described. Although thepresent method and system have been described with reference to specificexample embodiments, it will be evident that various modifications andchanges may be made to these embodiments without departing from thebroader spirit and scope of the application. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

The Abstract of the Disclosure is provided to comply with 37 C.F.R.§1.72(b), requiring an abstract that wilt allow the reader to quicklyascertain the nature of the technical disclosure. It is submitted withthe understanding that it will not be used to interpret or limit thescope or meaning of the claims. In addition, in the foregoing DetailedDescription, it can be seen that various features are grouped togetherin a single embodiment for the purpose of streamlining the disclosure.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate embodiment.

1. (canceled)
 2. A system comprising: a receiving module comprising oneor more computer processors configured to access target imageinformation representing a target image comprising a multiplicity ofimage pixels; a processing module comprising at least one computerprocessor configured for processing the target image information togenerate multiple image vectors for the target image by performingoperations comprising: for each one of multiple image portions formingpart of the target image, identifying a respective plurality of pixelblocks together making up the image portion, each pixel block being madeup by multiple contiguous image pixels of the target image, the multipleimage portions being substantially consistent in size and shape, and thepixel blocks being substantially consistent in size and shape throughoutthe multiple image portions; for each of the pixel blocks, assigning tothe pixel block an index value based on a visual attribute of the pixelblock; and for each one of the multiple image portions, compiling arespective index vector comprising the index values of the plurality ofpixel blocks making up the image portion; a comparison module comprisingone or more processing devices configured to perform a similaritycomparison between the target image and one or more candidate imagesbased at least in part on the multiple image vectors of the targetimage; and a result module comprising at least one processing deviceconfigured to output on a computer device a comparison result based onresults of the similarity comparison.
 3. The system of claim 2, whereinthe image portions are substantially similar in shape to the pixelblocks, but differ in size from the pixel blocks.
 4. The system of claim3, wherein the size of the image portions is a whole number multiple ofthe size of the pixel blocks, so that each image portion is completelymade up by a whole number of pixel blocks.
 5. The system of claim 3,wherein each image portion is made up by a square grid of pixel blocks.6. The system of claim 2, wherein at least some of the multiple imageportions overlap, with each pair of overlapping image portions sharingat least one pixel block in common.
 7. The system of claim 6, whereinthe processing module is configured such that the operations forgenerating the multiple image vectors comprise a sliding window methodin which a sliding window defining the size and shape of the imageportions is successively moved over a grid of the pixel blocks making upthe target image, to identify each possible image portion consisting ofa grid of pixel blocks fitting wholly within the sliding window.
 8. Thesystem of claim 2, wherein the processing module is configured to assignindex values to the pixel blocks such that the index value for eachpixel block corresponds to a statistical average of respective attributevalues of the multiple image pixels of the pixel block.
 9. The system ofclaim 2, wherein the visual attribute is an average of the lightintensity values of the multiple image pixels of the pixel block. 10.The system of claim 2, wherein the processing module is configured toperform one or more preprocessing operations on the target image toprovide a preprocessed target image, the multiple image vectors for thetarget image to be generated based on the preprocessed target image. 11.The system of claim 10, wherein the one or more preprocessing operationscomprises an edge detection operation.
 12. The system of claim 11wherein the one or more preprocessing operations further comprises anormalizing operation performed after the edge detection operation. 13.A method comprising: accessing target image information representing atarget image comprising a multiplicity of image pixels; in an automatedoperation using a processing module comprising at least one computerprocessor configured for processing target image information, processingthe target image information to generate multiple image vectors for thetarget image by performing operations comprising: for each one ofmultiple image portions forming part of the target image, identifying arespective plurality of pixel blocks together making up the imageportion, each pixel block being made up by multiple contiguous imagepixels of the target image, the multiple image portions beingsubstantially consistent in size and shape, and the pixel blocks beingsubstantially consistent in size and shape throughout the multiple imageportions; for each of the pixel blocks, assigning to the pixel block anindex value based on a visual attribute of the pixel block; and for eachone of the multiple image portions, compiling a respective index vectorcomprising the index values of the plurality of pixel blocks making upthe image portion; performing an automated similarity comparison betweenthe target image and one or more candidate images based at least in parton the multiple image vectors of the target image; and outputting to acomputer device a comparison result based on results of the similaritycomparison.
 14. The method of claim 13, wherein each image portioncomprises a two-dimensional area which is square in outline and is fullymade up by a square grid of pixel blocks contained within the imageportion.
 15. The method of claim 13, wherein at least some of themultiple image portions overlap, with each pair of overlapping imageportions sharing at least one pixel block in common.
 16. The method ofclaim 15, wherein the processing of the target image informationcomprises performing a sliding window method in which a sliding windowdefining the size and shape of the image portions is successively movedover a grid of the pixel blocks making up the target image, to identifyeach possible image portion consisting of a grid of pixel blocks fittingwholly within the sliding window.
 17. The method of claim 13, whereinthe assigning of the index values to the pixel blocks is such that theindex value for each pixel block corresponds to a statistical average ofrespective attribute values of the multiple image pixels of the pixelblock.
 18. The method of claim 13, further comprising performing one ormore preprocessing operations on the target image to provide apreprocessed target image, the multiple image vectors for the targetimage to be generated based on the preprocessed target image.
 19. Themethod of claim 18, wherein the one or more preprocessing operationscomprises an edge detection operation.
 20. The method of claim 19wherein the one or more preprocessing operations further comprises anormalizing operation performed after the edge detection operation. 21.A non-transitory computer readable storage medium having stored thereoninstructions for causing a machine, upon execution of the instructionsby the machine, to perform operations comprising: accessing target imageinformation representing a target image comprising a multiplicity ofimage pixels; in an automated operation using a processing modulecomprising at least one computer processor configured for processingtarget image information, processing the target image information togenerate multiple image vectors for the target image by performingoperations comprising: for each one of multiple image portions formingpart of the target image, identifying a respective plurality of pixelblocks together making up the image portion, each pixel block being madeup by multiple contiguous image pixels of the target image, the multipleimage portions being substantially consistent in size and shape, and thepixel blocks being substantially consistent in size and shape throughoutthe multiple image portions; for each of the pixel blocks, assigning tothe pixel block an index value based on a visual attribute of the pixelblock; and for each one of the multiple image portions, compiling arespective index vector comprising the index values of the plurality ofpixel blocks making up the image portion; performing an automatedsimilarity comparison between the target image and one or more candidateimages based at least in part on the multiple image vectors of thetarget image; and outputting to a computer device a comparison resultbased on results of the similarity comparison.